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DETAILED ACTION 

1. The Appeal brief filed May 5, 2005 is considered and claims 1-12, 14-19 and 21- 
24 are pending. 

2. The rejection of claims 1-12, 14-19 and 21-24 under 112, first paragraph is 
withdrawn. 

Drawings 

3. This application has been filed with informal drawings which are acceptable for 
examination purposes only. Formal drawings will be required when the application is 
allowed. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-12, 14-19 and 21-24 are rejected under 35 U. S. C. 103 (a) as being 

unpatentable over U.S. Patent # 5,767,842 ("Korth") in view of U.S. Patent # 6,407,679 

("Evans et al.")- 

As for claimsl, Korth teaches a camera Fig.1 (2) corresponding to one or several 
cameras, a computer side of the interface there need only be a way to detect the 
fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to one 
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or more memories with CPU connected to the cameras, and processes running in the 
CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. Accordingly, Korth 
teaches all the claimed limitations as recited in claim 1 with the exception of providing a 
classify groups of gestures associate each of the classes with one or more possible 
keys, assign a probability to each of said possible keys, and integrate the probabilities 
assigned to the possible keys to identify a word for a sequence of gestures. 

However, Evans et al. teach (Fig. 1), where each key, or finger, would represent 
several different alphanumeric characters (column 2, lines 40 and 41) and a glove 
usable in a virtual environment where each finger has a group of letters associate with, 
after running through steps 400 through 1400. the system determines the most likely 
word is "hi" (column 8. lines 1 9 and 20) and matching the key words with one or more 
words in the dictionary, generating all possible permutations of word seouences. and 
selecting the most probable word seguence or partial sentence (column 83. lines 24-27) 
corresponding to classify each gesture into one of the plurality classes depending on 
the movement of the gestures and associate each of the classes with one or more 
possible keys, assign a probability to each of said possible keys, and integrate the 
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probabilities assigned to the possible keys to identify a word for a sequence of gestures. 
It means that each key represents a classification of several alphanumeric characters. 

It would have been obvious to utilize the finger labeled as taught by Evans et al. 
in the user command system disclosed by Korth because this would allow the user 
command to match each key sequence with one or more words in the dictionary. 

As for claim 2, Korth teaches a keyboard that should provide the user with the 
key information and feedback. Tactile feedback of virtual keyboard is given by the 
contact of the finger-tip with the working plate (column 7, lines 14-17) corresponding to 
a feedback is provided to the user on what kind of keys are associated with the user's 
gestures. 

As for claim 3, Korth teaches a visualization of the actual key function allows the 
handling of keyboards with multiple shift keys and subgroup display for large characters 
sets (column 3, lines 1-4) corresponding to displaying keys on the display, a keyboard 
for musical instruments (column 3, line 11) corresponding to playing sound labels for 
keys and playing special sound indicators, a video sensor that is used for monitoring the 
keyboard to generate image data representing positions of an operator's hand with 
respect with the keys (column 3, lines 55-58) corresponding to displaying image 
indicators on the display, a camera Fig. 1 (2) corresponding to projecting the keyboard 
to any surface and displaying picture of the keyboard with the user's hand. 

As for claim 4, Korth teaches FIG. 2 shows the area viewed by camera 2. In the 
observed region 4 the user's hands are detected. The enlarged section 5 of FIG. 2 
represents an area of the camera's image which is chosen for further image processing 
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corresponding to one or more cameras, another image evaluation procedure then helps 

the image processor to match the virtual keyboard with the template. The place-mat 
may be applied on a piece of soft tissue (e.g. polyurethane foam) to reduce the strain on 
the finger-tips from the impact on a rigid desk plate (column 7, lines 47-52) 
corresponding to one or more memories with CPU connected to the cameras, and A set 
of simple intuitive 'gestures' may be defined to control the operation of a virtual 
keyboard (column 6, lines 7 and 8) corresponding to processes running in the CPU that 
associates gesture movements with typing and produce gesture associated with textual 
output and a gesture capturing module that captures gestures through camera sensors, 
and the shape of the hands can be used to send gestures with additional information to 
the optical interface (column 2, lines 55 and 56) corresponding to an integrator module 
that integrate sequence of candidates of most probable keys into unique output key 
sequence. 

As for claim 5, Korth teaches a method of user command or data acquisition in a 
processing system that allows for analysis of human hand and finger motions. The 
activities are then interpreted as operations on a physically non-existent input device. 
The resulting optical user/computer interface complements the visual screen-to-user 
path with a hand-to-image data acquisition link from the user to the processor unit. It 
allows for the emulation of different types of manually operated input devices (mouse, 
keyboard, etc.). Mechanical input units can be replaced by virtual devices, optimized for 
the current application and for the user's physiology. The speed, simplicity and 
avoidance of ambiguity of manual data input will be maintained. Sensing of finger 
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motions can generate feedback to reduce the hazard of repetitive strain injury (RSI) 
syndrome (abstract). 

As for claim 6, Korth teaches a method of user command or data acquisition in a 
processing system that allows for analysis of human hand and finger motions. The 
activities are then interpreted as operations on a physically non-existent input device. 
The resulting optical user/computer interface complements the visual screen-to-user 
path with a hand-to-image data acquisition link from the user to the processor unit. It 
allows for the emulation of different types of manually operated input devices (mouse, 
keyboard, etc.). Mechanical input units can be replaced by virtual devices, optimized for 
the current application and for the user's physiology. The speed, simplicity and 
avoidance of ambiguity of manual data input will be maintained. Sensing of finger 
motions can generate feedback to reduce the hazard of repetitive strain injury (RSI) 
syndrome (abstract). 

As for claim 7, Korth teaches a method of data input using video sensors for 
monitoring positions of an operator's hand with respect to keys on a virtual keyboard 
optically produced on a surface. 

As for claim 8, Korth teaches a camera Fig.1 (2) corresponding to one or several 
cameras, a computer side of the interface there need only be a way to detect the 
fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to one 
or more memories with CPU connected to the cameras, and processes running in the 
CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
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associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 9, Korth teaches a camera Fig.1 (2) corresponding to one or several 
cameras, a computer side of the interface there need only be a way to detect the 
fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to one 
or more memories with CPU connected to the cameras, and processes running in the 
CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 10, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
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associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 1 1 , Korth teaches a keyboard that should provide the user with the 
key information and feedback. Tactile feedback of virtual keyboard is given by the 
contact of the finger-tip with the working plate (column 7, lines 14-17) corresponding to 
a feedback is provided to the user on what kind of keys are associated with the user's 
gestures. 

As for claim 12, Korth teaches a visualization of the actual key function allows the 
handling of keyboards with multiple shift keys and subgroup display for large characters 
sets (column 3, lines 1-4) corresponding to displaying keys on the display, a keyboard 
for musical instruments (column 3, line 1 1 ) corresponding to playing sound labels for 
keys and playing special sound indicators, a video sensor that is used for monitoring the 
keyboard to generate image data representing positions of an operator's hand with 
respect with the keys (column 3, lines 55-58) corresponding to displaying image 
indicators on the display, a camera Fig. 1 (2) corresponding to projecting the keyboard 
to any surface and displaying picture of the keyboard with the user's hand. 

As for claim 14, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
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one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 15, Korth teaches a set of simple intuitive "gestures' may be defined 
to control the operation of a virtual keyboard (column 6, lines 7 and 8) corresponding to 
means for associating gesture classes with individual typing keys. 

As for claim 16, Evans et al. teach the process loops back to select another 
word. This process continues until all the words have been placed in the data structure 
(column 4, lines 62-64) corresponding to providing training data in words or sentences 
with certain timing data. 

As for claim 17, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
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From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the. keyboard were presented. 

As for claim 18, Korth teaches a keyboard that should provide the user with the 
key information and feedback. Tactile feedback of virtual keyboard is given by the 
contact of the finger-tip with the working plate (column 7, lines 14-17) corresponding to 
a feedback is provided to the user on what kind of keys are associated with the user's 
gestures. 

As for claim 19, Korth teaches a keyboard that should provide the user with the 
key information and feedback. Tactile feedback of virtual keyboard is given by the 
contact of the finger-tip with the working plate (column 7, lines 14-17) corresponding to 
a feedback is provided to the user on what kind of keys are associated with the user's 
gestures. 

As for claim 21 , Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 



Application/Control Number: 09/603,980 Page 1 1 

Art Unit: 2674 

motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 22, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 23, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
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motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

As for claim 24, Korth teaches a camera Fig.1 (2) corresponding to one or 
several cameras, a computer side of the interface there need only be a way to detect 
the fingertip motions of the keyboard operator (column 4, lines 27-29) corresponding to 
one or more memories with CPU connected to the cameras, and processes running in 
the CPU that associates gesture movements with typing and window may appear only 
during the input of data (column 4, lines 54-55) corresponding to the produce gesture 
associated textual output. A virtual keyboard 3 serves as data or command input device. 
From the computer side of the interface there need only be a way to detect the fingertip 
motions of the keyboard operator (column 4, lines 25-29) corresponding to sensing the 
typing gestures. It is inherent in the art that the gestures are associated with the most 
probable keys that would be typed if the keyboard were presented. 

Response to Amendment 

Applicant's arguments filed May 5, 2005 have been fully considered but they are 
not persuasive. Applicant's representative argued that the prior art, Korth, does not 
teach a invisible virtual keyboard. Examiner disagrees with the applicant because the 
prior art, Korth teaches in the abstract The activities are then interpreted as operations 
on a physically non-existent input device" and the keyboard needs not to exist 
physically (column 4, line 25) corresponding to a invisible virtual keyboard. The 
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applicant argued that the prior art, Evans et al., does not teach "the computer 
processes, those finger movements are transformed into text and classify the finger 
gestures into classes, and associate each of these classes with one of the keys of the 
keyboard". Examiner disagrees with the applicant because even if the prior art does not 
teach word for word the limitation claimed but the concept is inherently there. The prior 
art teaches whereby each key, or finger, would represent several different alphanumeric 
characters (column 2, lines 40 and 41) where the examiner is interpreted in Fig.1 as 
each key or finger represents different classes of gestures. And it says again further 
down in the prior art that the left-hand ring finger, for example, could represent "s", "v", 
and "x", the right-hand index finger could represent "j", "u," "y", "h" and "n", and so on 
for each finger (column 2, lines 43-46) where the left-hand ring finger represents a class 
of gestures and the right-hand index finger represents another class of gestures. The 
applicant argued that the prior art does not teach "each word is identified on the basis of 
the probabilities assigned to the possible keys assigned to the gesture classes that 
make up that word" but examiner does not agree with the applicant because the prior 
art, Evans et al., teaches after running through steps 400 through 1400. the system 
determines the most likely word is "hi" (column 8. lines 19 and 20) and matching the kev 
words with one or more words in the dictionary, generating all possible permutations of 
word seguences, and selecting the most probable word seguence or partial sentence 
(column 83. lines 24-27) corresponding to "each word is identified on the basis of the 
probabilities assigned to the possible keys assigned to the gesture classes that make 
up that word". The claims have been amended but not enough to differentiate over the 



Application/Control Number: 09/603,980 



Page 14 



Art Unit: 2674 

prior art. They need to be further amended to overcome the prior art of record. 
Therefore, the rejection is maintained as was rejected in the previous office action. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jean Lesperance whose telephone number is (571) 
272-7692. The examiner can normally be reached on from Monday to Friday between 
10:OOAM and 6:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard, can be reached on (571) 272-7603. 

Any response to this action should be mailed to: 
Commissioner of Patents and Trademarks Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 
Jean Lesperance 
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